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(54) Locating the position and orientation of multiple objects with a smart platen 



(57) The present invention is an intelligent scanning 
system (20) for processing a digital input image to au- 
tomatically characterize a plurality of objects therein. 
The system then employs the characterizations as the 
basis for rudimentary image editing operations so as to 
produce a digital document. In the digital document, the 
objects may be derotated, shifted, cropped or otherwise 



aligned in a predetermined fashion in accordance with 
a template. The scanning apparatus (26) of the present 
invention not only enables the scanning of a plurality of 
objects, but does so in an intelligent nnanner so as to 
enable further processing and manipulation of the im- 
ages associated with the objects to create an output 
document. 
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^ \j Description"-".:^:^- '^i ^Coi^ > > j: ..^i'.-/'' 

' ' • ' [0001] Thts.inverntion relates' g^neraliy-tb an iniage'' input-' System; arid more' particularly to the automatic characteri- 
zation and editing of a plurality of irnage objects placed oh' the platen of an image input device. 
5 [0002] To improve the efficiency of scanning systems and digital copiers, it is desirable to produce a single scanned 
image composed of several separate^objecfs'(e.g;/'phd6graphs)ia1d side' by side, ^ pf'eferably not touching, on the 
platen of the input scanner. To facilitate automated editing of the bbjects?it is necessary to identify the position, shape 
and rotatiori' angle- of each'object. Such a capability c^n erihance productivity by decreasing the time required for 
scanning multiple objects and by automating rudimentary editing operations. 
10 [0003] Heretofore, =a number of patents and publications have disclosed image segmentation and structured images, 
■ - : the:relevant pdrtibns>of which fnay be briefly s^u^ ' ' ' ■ 

. ••[0004].^ US-A-5,485,568 to Venable et'ar.,-is~kued Jahua'Fy^ 16; i 996, and ^hereby iricorporated tiy reference, discloses 
tj ; a method and apparatus for representing a ebnnplex-color radteV irnage'- as a collection of objects in a structured image 
' ^-fGrmat^^'aihiSrarchicaf; device-iridependent^ document, generated using the techniques 

.' descfibed-by Venable^is'a represehtatioh.of data that may be 'rendered into a raster innagb.' The data includes simple 
■o- > raster images as vvelK^s a-hier-arc The possible data 

'.^ types for objects in th6 struetur^d'inhage'rnclijd^ a reister image, text, graphics, image processing description, and files 

containing multiple image representations. 
: [0Q05] In "MANAGING 'AND REPRESENTING' IMAGE 'WORKFLOW IN PREPRESiS' APPLICATIONS", Technical 
20 Assoeialion of the Graphic "Arts (TAGA) Vol.' i, 1995 "Proceed ingV pp.' 3 hereby incorporated by reference for 

its teachings; Venable -et al. te^ch th d 'use/ of structured imagies tb mariage prepress workflow. Ah operation such as 
gang scanrtThg is described as a means for capturing sevemf photographs roughly alignbd on a scanner platen. 
[0006] In accordance with the present invention, there is provided an imaging apparatus, including: 

25 an imago input device, said image input device producing a digitized image including representations of each of 

a plurality of objects imaged by said device; 
^ ■ ^ ' ■ a programrtiable computer capable of processing the digitized image, said corriputer including a first memory for 
storing at least a portion of the digitized image and program memory for the storage of executable code suitable 
for causing said computer to execute image processing'bperations on the digitized image, 

30- ^said-cdmjputen rn accordance with prfeprogranhnned instruetionsv identifying the plurality of objects within the digi- 
tized input image, modeling shapes representing boundaries of each of the plurality of objects, and characterizing 
each of the plurality of objects by' pa'rarineters including shape, position and orientatibn; and 
said computer automatically composing an output document including a representation of at least one of the plu- 
rality of objects. 

35 ,- ■ • ' ' ■ ■ 

[0007] In accordance with another aspect of the present invention, thiere is provided A digital copying apparatus, 
■ Including^ ' - " ' ■ 

■ ■ " ari image input device-having a transparent platen 
40 representations of each of a plurality of objects placed on the platen; 

-: ■ a programnrtable'^cbmputer capable of controlling the operation of the digital copying apparatus and of processing 

; • : ■ ; ^the di^itiz^'d i'magevsaidconriputer including a first memory for storing at leaM a portion of the digitized image and 
program memory for the storage of executable code suitable for causing said computer to exep^ute image process- 
• • ' r : ing operatidh's'oh^ digitized im^ge,'said computer, in accordance with preprogramnnedMnsfru^ identifying 
thepruraNty'Of 'objebts within' thi digfti^ed Trip ut inn age, modeling shapes repfesehtihg boundaries of each of the 
-■^ ■ plurality of %'bjects /and characterizing each of the plural i^^ objects iD'y'par^.ametersJhcluding shape, position and 
orientation ^ahd 'said' computer automatically cbnripbsirig an output docUrrient including a representation of at least 
one'of the plurality of' objects; ■ ' . ^ 

a user interface, said user interface comprisirig- a disj^lay ' tb 'depict th6 bUtfiiui document' arici ^ plurality of user 
selectabr(^:bpti6n, whei*4ih said computer fuilher 'includes preprogranrihnedm'st ructions to update the display in 
^ K accbi^dance^ith'-a selection' of the uis'er selec^^ ' • 

a printing engine to produce, in accordance with the output'. document, -a substrate bearihg'irriarks in accordance 
•-^■=with' representations cbh ' ' [. '"' ' ' 



55 



[0008] The present 'invent ioh isrdirected'to a 'sysferti ihterided to accomplish the autbnaattc.'^det^^^^ of inde- 

perideirit re'^giohs or segments for objects within a'scaWr^ed ir^i'age;Th^^ ihveritibn further provides a' user interface and 
document templates tolacilitate the automatfc plaW^ment bt the plui-ality of objects atpds it i'bns within a digital document 
so that -the document may be rendered, stored or '^j-ah^mitted! Thus, thei present iriver^tion combines a number of 
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graphics and image processing techniques into an automated, user-friendly application for productivity enhancement. 
The application can enhance productivity by decreasing the time required for scanning multiple images, by automating 
corrections for alignment of nriultiple images, anqJ,even automatically placing multiple images in the document template. 
[0009] The present invention accomplishes these objectives by:. : . . 

1) locating a plurality of independent objects vvithin the image - . . 

2) mcdeling the shape _ofJhe identified objects :(e.g.. rectangle) - . , 

3) creating a structured image description. ideritifying the location, shape and orientation of each object within the 
image. ^, . - ■ . 



[0010] One aspect of the invention deals with a .basic problem in digital. image processing,:that of identifying plural 
objects within a digitized image. The solutipn tqthis problem is then leveraged.so as to enable the automated recognition 
and placement of objects to compose a digital document. The techniques described herein enable a user to expediently 
scan a plurality ordistinct documents or objects Jn.a-single scann ing. operation,, automat icatty.^separate those objects 
15 ; by recognizing them as independent objects within.the digitized image, and reconnppsing a digital. document using one 
or more of the recognized pbjects. Anotfier.aspect of the; present invention allows fqr-the automatic.creation of a struc- 
. , tured image representation of the digitized. image sath^at,the..image:Objects be-easily. extracted and further proc- 
essed, independently. , ' ^ ' 

[0011] The techniques described above are. advantageous because they improve the efficiency of a scanning proc- 
ess, allowing multiple original documents to.be scanned alone lime, as^welLas the document composition process by 
recomposing the documents. into a single.document using a predetermined template.. in addition, the techniques allow 
for automatically characterizing. physicaKattributes.^(e:g.,.!ocati^^^^ and orientation) of the objects without user 

intervention. , . . - . . . ■ c- ; ; -.^^ ■ ^. : , • , 

[001 2] An embodiment of the present invention will now be described with reference to the accompanying drawings. 
25 in which: . ... , . ^ . 

Figure 1 is an illustration of the equipment that forms an image processing system. sen/ing as one embodiment for 
the present invention; , , , . . 

Figure 2 is a block diagrann of the various components comprising. the. system of Figure 1; 
30 Figure 3 is a flowchart illustrati.ng the generaLprocessirig steps carried out on the system of Figures 1 and 2 in 

. accordance with the present invention; ' ' ... ,;. j.. . . ... 

Figures 4-7 are detailed flow charts illustrating the- processing steps .carried put in accordance with various em- 
bodimerits of the present invention; . . - . , . 

Figure 8 is an illustrative example of a portion of a digital document; 
35 Figure 9 is an illustration of the output of the system of Figure 1 when an input image is processed in accordance 

with the present invention; " - . - , - . 

Figure 10 is an exemplary user interface screen associated with one embodiment of the present invention; 
Figure 11 is an example of an image containing multiple objects obtained by a scanning operation; and 
.. Figure 12 Js a comparative illustration of deskewing and derotation.operations as applied to an image segment. 



[001 3] , For a general understanding of the present invention, reference is made to the drawings. In the drawings, like 
reference numerals have been .used throughout to designate identical elements. In :describing the present invention, 
the followirig term(s) have been used in the description. m . t,... . - . 

' [f^^ffl c.T^^^^^/'rn i^dat^ signals that indicate or include iriformatipn. When an item of data 

. pari indicate one of a nymber.of possible alternatives, the item qf data .has^ one of a, number of -values." For example. 
4PP^^ '^^^ referred to as a "bit, ".has one of lwo values,: interchangeably referred to as "1 - and "0" or 

"OKf." and.'QFF" or/high" and "low." A bit.is.an "inverse" of another bit if the two,t>its. have different values. An N-bit 
ileni of data has one of 2N values. A "mulli-bit" item of data is an ilem of, data that includes more than one bit. 
^ , "Memory circuitry" or "memory" is any circuitry that can store data, and nriay include. local and remote memory 

■^^^and inpyt/putput devices. Exanhples iriclude.semicpnductor ROMs. RAMs. and storagejriedium access devices with 
'data storage media that they can access. A 'mempry cellVis memory circuitry that can rStqre-a sicigle unit of data, such 
,as a bit pr other n-ary dig it pr an.analog value. . , ... 

[0016] A signal "indicates" or "selects" pne^of a set.pf , alternatives if the signal causes -the. indicated one of the set 
of alternatives to occur. For example, a signal can indicate one bit set in a sequence of bit sets to be used in an 
^ operation,. in. which case the sign^ causes the. indicated bit set to be used in the operation. ; : 
[^111 ^.^ An "'mage" is a pattern of physic^l!ight.^^im^ge.may include characters, words, and text as well as other 
features such as graphics." A text may be included in a set of one or more irnages. such as in images of the pages of 
a document. An image may be processed so as to identrfy specific "objects" within the image, each of which is itself 
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" . ■ ; ■ 'an irtiage;-A objfect may be* of'any size and shape'ahcf has "physical aJttributes"'or characteristics including, but not 
' . ■ ' Itrriited/to po'sitibnVs^^ o'i' . i? -r. -.'r. • v.. 

:o • J . : . ' '^'[0018] ^'' Ahitemof datrwiries" an iniag^ 

■■■ For example, -a two-8imehsioriarai^ray can define all or ahy fikrt of an imacfe, with each item of data in the array providing 
•s ■•■ a Value indicating tiie color lif a' respective loc'k^^ "■ 
^ -[OOIS] ' Ah itefih'of data "defines^^^^^ 
• :■ ^ images in'ihe se^"'' ■ ' ■' " • \' ' ' ' 

■ ^- ' [0020] ' Eai'ch^ldcati6ri in an innag^e may be c&li^d a lh\an array defining an image in which each item of data 

" ■■ ' provides a v^lijef each- value' ihdicating the 'color of '^'location 'rna/ bei'talled a"*pixel value". Each pixel value is a bit 
10 " Tn ai "binary' fbrnn'' of-^n'irtiaige, a '^ray sciale'vafue'iri'a ^gra^ scale form" of an imagefor a set of color space coordinates 
in a "color coordinate form" of an image, the binary form, gray scale form, and color coordinate form each being a two- 
- ■ -'dimensional array defining ^n image. ' ^ ^ * • ■ 

• ■ -tOOSi] ' ■'Ah'opermion perfoirhs' "inriagepi-oce^sing"' when it of ^ itenn of data'that relates to part of an image. 
. . [io(jj22!]' Pixels are- "neighbor^" 6f ''neighborihg"* within an image wh'en there ai-e no other pixels between them and 

" ■ ' • 75 ' ■'^ttiey meet ah apprdpMt^-criterionl^ neighboring, if the 'pixels arb reciangular and appear in rows and columns within 
' ' ' ■■ ' a tw^o-dimerisionaMfti'a^e;^ hk^f&'A 6t~8 neighboring pixels, depending 'oh the criterion used. 

. ; [0023] An'""edge" occurs'in animage "Wheri-fwo heighbqrihg pixels liave' sufficiently different pixel values according 
;■ ' to an appropriate ci^iteribnter the' occOrr^^^ edge bet we eri th^m. The.terrns "edge pixel" or "boundary pixel" 

■' nnay be applied to bri'^ br both of -tw^^ 
- 20 '' [0024] ■ An'"image chardclerislic" br'^characterisllc" is a measurable attribute of ah image. An operation can "measure" 
a characteristic by producing data indicating the characteristic us in g^^^ defining an image.' A characteristic is meas- 
ured "for an "ihiage" if the characteristic is measured in a mahrier that ^is likely to produce approximately the same result 
'each time it occurs.' ' ' ^- '' ' " > "^='" • 

' ' ■ ^[0025]^ 'A "version" of a fii-st image is a secohd image ^ro^iuced using an item of data defining the first image. The 
' '2S' sccdnd imagc may be idGnticai-tb thG'first image, or if may be. modified by loss of rosolutiori, by changing the data 
' ' ■ defining the fir^st irhage, or by other processes that modify pixel values of the first image. 

[0026] An "image input device" is a device that can receive an image and provide an item of data defining a version 

• ' , of the inniage. A "isbanner" ik an image input device that receives an image by a scanning operation, such as by scanning 
' ' 'a' document. -A scanner may haye a transparent surface (platen) or equivalent means to support a document during 
' .^^ scanning. Other^wie^li-knowh trriage input devices include digital' cameras, facsimile machines, and video recorders 

having the capability to store da^^^^^ representative of the intensity oflight refiected from the surface of objects 

"• - ' at which the ^device Is'di rented:' " ' 

[0027] An "image output device" is a device that can receive an item of data defining an image and provide or render 
the image as output. A "display"' is an image output device that provides the output image in human viewable form, 
■ 35 arid a "printer" is an image output device that renders the output image in a human viewable,. hard copy form. 

' [0028] Referring now to Figures 1 and 2, depicted therein is a smart platen system 20 ir» which the present invention 
finds particular us^. System 20 includes a computer 22 capable of receiving digital data representing an image of an 
original document 24 placed upon a platen of scanner 26. Computer 22, initially stores the digital input data from 
scanner 26 in men]ory,;52 (e.g., RAM or magnetic disk storage) where the image may subsequently be accessed. In 
40 addition to the digital data; merinory 52 may also include program memory for the storage of object code suitable for 
^ birecting the processor to execute image prpcessing operations in accordance with the invention, described herein. 
; Computer 22'HaS_asi?b^^ therewith a user interface (U/i) 28 including one or'mpre user input devices 30, such as 
" ' "a 'keyboard; a keypad^a rtibuse', trackball stylus or equivalent pointing device, etc. 

[0029] Also part Of system 20 is an.jmage output^device such as printer, 34 which nhay rnclude,ajaser-driven. xero- 
^ ^ 9''^P^'c printing'ert'gihfe''as'fburib1h*a nurinber p^cohimercially' available printers. . In a preferred embodiment, system 
"' '[ 20 1s' 'emblbyed digital image bata received as, input" fr;om/^ scanner ^26,' utilizing, image processing 

' '"^ ■ ' ' sbftware running in prbcessor 50, so as to produce an output file that may 6'e rendered by printer 34, stored in memory 
50, and/or transmitted to another device via network 40. 

[0030] Although system 20 is depicted as a plurality of interconnected units, it will be appreciated that the system 
so may also comprise what' is.commpnly referred. to as a "digital copier". In such an embodiment, the components of 
system 20 are integrated within a common housing, and the user interface (including display and input devices) may 
, , have a significantly different appearance. An example of such color digital copying syste.m.lr) . which the present 
' / ' \ ' i Invent ion^ nriay ' fi nd pa rt 1 c u la r use Is the Xerox 5775 D i gital dojq r' bop^j © r, . cbnt rolled via a , touch xic reen co 1 or video 
' ^' ' ' ' mbhitof that shows the progressioni of operations. Jit will be a^p^ .that the document placed ^ upon the scanner 

J...-' ' '|55^ - ^ilaten'fn'systerii 20 prefb^^^ jnclQd^sa piu/a^^^ objects represented by niarks on a substrate 

surface, so that such object^ may be scannetfby^a slng^ 

to which the following description will be directed is a single stcanned image representative of . several separate pho- 
• tbgraphs laid side by side on the platen of scannerYe',' but not touching or overlapping'. In accbrdance with the present 
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invention it is 'desired to automatically identify the position, shape.and rotatbn angle of each original photograph. 
[0031] Given an input innage generated by scanning several separate photographs laid side jDy side on the scanner 
platen, the present invention automatically identifies at least the positiori. shape and orientation angle of each photo- 
graph. As shown in the flow chart of Figure 3. the process carried put by computer 22 during the processing of the 
input image includes three general steps. First, at step 100 the.olDjects within the image are located and boundaries 
of the object are generally ideritified Pnce the objects are, located.. the shape of the objects is modeled at step 200. 
Having located the objects and modeled their shape, a structured image representing. the image and objects therein 
can be created as represented by step 300. The structured image preferably ^includes data representing not only the 
image data itself, but^data representing the Ipcationjshap^ or orientation of each object, or some combination thereof. 
Alternatively, the output may 'be a page description language format or equivalents formats suitable for storing the 
image informationMn a retrievable form.' ■ ^.^ . . ^ . , 

[0032] In a preferred embodiment of the present invention, the scanned input image, (or a lower resolution version 
thereof) is loaded into a memory franrie buffer (RAM). where it is analyzed in accordance with the previously described 
steps. For purposes of the following detailed description, it is assurned.that objects dp not occlude one another and 
that the background of the image is contiguous, these siojiplifyTng assumptions are intended for purposes of explanation 
only and are not intended as Mmitatipris of the'inventiori.pne skilled in thaart wilj appreciate that \t)e invention described 
herein is exterisible so as pot to require operation . on ly.wittijn tt^e.b^^ of such assumptions. 

[0033] As depicted by the flow chart pf .Figure 4,.thepbjecf location -step 10 is performed "by first identifying the 
background region of the input image 102, characterizing the and.then using the characteristic 

of the background region as a seed, jderitiiying. all the pixel? represeriting the backgrpund region with an adaptive seed 
fill algorithm 106. Background pixels are pixels not associated with any objects, or more simply, they are pixels repre- 
sentative of those regions lying outside of the objects, the values of which are controlled by the "background" against 
which the objects are placed during scanning (e.g., the underside of the platen cover). One embodiment employs the 
average color of a small region in the upp^r left-hand corner of the spanned image as an; initial estimate of the back- 
ground color. Alternatively, other sampling ppcrations.nnay be cnnployod to deterrnine the background color as de- 
scribed, for example, in US-A-5,282,091 for a Programmable Apparatus for. Determining Doc umGnt. Background Level 
by Farrell. . , . 

[0034] Once the background color is characterized at step 1 04, an adaptive algorithm is preferably applied to monitor 
the background color and to accurately identify the objects. An example of. a seed fill algorithm suitable for use in the' 
present invention is described in Graphics Gems I. A. Glassner Ed.. Academic. Press, pp. 275-277, 1990, hereby 
incorporated by reference. An adaptive algorithm is required because.the background pixels.may have significant color 
variation resulting from a variation in illumination over the platen area. The. adaptive seed fill algorithm is applied to 
the scanned color image data using an initial seed point characterized by the background, for example, the upper-left 
corner of the image. Generally, the adaptive seed fill algorithm 'fills a binary, frame buffer with a^mask indicating all 
contiguous pixels identified as background pixels. In a simple embodiment, represeritec^ by step 112, a pixel is consid- 
ered to be a background pixel if its color falls within a small distance. e of the current average background pixel value. 
This distance is calculated as an Euclidean metric in red, green, blue (RGB) color space i 

40 ^^-SQRT((P^-AdAvg/V(Pg-AdAvgg)^+(P^:AdAA^ 

where P^, AdAvg^ are, respectively, the RGB components of the pixel under, tesj.ancLlhe^a^^^ background value. 

and is the distance measurement. The value of £ is fixed and empirically determined in pne^ embodiment. The test 

conducted at step 112 is: * ^ = < " V . / ' .^i 

■45 [CX)35] if d<e, then pixel P is a background pixel, else pixel P i s a fpregroiirid pixQl. V , •* J f/ 

[0036] The average background coloi^ is adaptively modified"at'step J14 by. taking! the average ,v^ of the last N 
■ pixels that have been classified as background. For efficiency, 'the system preferably.'c^^^^ average 
' using the equation: ' . . V- 
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- ■ / AdAvg' 3 (N*AdAvg-AdAvg+LastVal)/N, - 

where AdAyg' is the modified average. AdAvg is the previous adaptive average. LastVa/js tlie. value of the last pixel 
identified as background, and fsl is the averaging window Clearly, this is not a true, running average, but it tracks the 
running average adequately and is nniore computatipnally efficient than a strict, running average calculation. Altema- 
tiveiy. the value of e can be adaptively modified. For example, E might be based on the standard deviation of the last 
several pixels' identified as background. /etc.'- ' . ' 

[0037] It will b'e appreciated that alteVnative ririethods for the detection of background regions may be employed, and 
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' ' :^'-* '-.'-i; - ■ ■ that'the proc^ss'of'bbtaihihg a binary map^distinguishM^ the dbj^^^^ the background may be accomplished using 
. ■: - • ' A simplifliedthreshdtding operation basted upon tHe'tDackgrbUiid cobr. In a preferred embodiment, the background color 
:.. V. V - may be a function of the' color of the platen co\/er,'=sUch' as a'White cover. As a further example, the 5775 Digital Color 
Copier employs a spectrally reflective platen cover that P^sLilts in a bluish-backgrbund color that may be detected and 
■ 5* ■distinguished frohi' the objects. ■ i. --v . 

; ^ [0038]*.' Having fdentified all tekgro^ 

■^'^ ■ - ' , the process at' step 1 20 1^ ekecuted tb si niboth noisy edges iri the bsi'ckgrOun'd mask using morphological fi Itering. More 

- t r . '^v ' K.. "specifically.'a' morphbbglcarc^^ filter is pref6Yafely'appried background mask'to eliminate single pixel noise 

and'to srilooth object'ed^es/ Subsequently/ cdhtigi^ are Ideated, step 122, thereby identifying 

- * ' 1.0' ■ • the objects. Objects'-a re "idiBht if i^d-- by scan nin g^ the backg rou nd ' nids k ge n e rated^ by' th e adapt ive seed fi 1 1 op e rat ion 

■ ' - - ■ * (step l06)PStarting witH'the iipper left hand pixel ;4h6 mask i^^^^^ in'^a scah line fashion for a pixel not classified 

' '-^ ' ■ in the'mask^as a backgi'ourid pixel - thus idehtif^ihg jiiixels^assodat^ object? The use of the seed 

v. ,: -; - ' i--^ fill argorith'm for ^identifying t^^^ are closed, "or in 'othdr words, complete 

' ^TDOundary is 'formed 'ab(^^^^^ • 
. / . 75' ^-"^ [ObSSJ-'^' -'At 'step 'f 2-^;; the 'boundary of ^ri object is identlfied by tracing its edge. The boundary of the foreground object 
M : - ; ' 'Is' t?ac'e'd usirig' a sii¥iple B^cdhnected ■ te'dg^'^traversal Which provides - an ordered-set 'of ' pol nts t racin g th e edge of the 
' - object. Buch ari' edge'tra\;f^re^ 

" ' ^sihnilar to'^'word or character based\recdgWiti6H^^^ is described, for example, by R. 

Bozinovic et al. in "Off-tjihe fcuriSiVe Scripl-Wdrd Reddgniti'dn", IEEE Transact ions bh Pattern Analysis and Machine 
Intelligence; Vol. 11\ No.' 1"(January-1989X"Once-t^ all pixels associated with thebbjeci in the mask 

are marked as background so they will not be processed a second time, the object is added to the foreground object 
^ - list and thdn the scarih trig of step' T22 is continued asMndicated by 'test step 1 26. Subsequent to completing the fore- 

" ' ground scanning to' identify all obj^dfs, a" review of the identified dt:jeCts may be' completed 'as represented by step 
■■'130rin many cases, the scanned image may contain undesirabtdi^ objects;' such objects can be eliminated 

2S'' , froin the object' list at this' step. In oho ^rhbodimont: the^ review of the object list may simply eliniihato small objects as 
, : . .. . . unlikely images; For exfample, in a scan of a yearbook page each' ihnage has associated with it a text caption that is 
not to be classified as image dat^.-Such bajDtidns consist of many snriall perimeter objects^ so that by measuring the 
' ■perimeter length of the traced edges; it is' possible to eliminate bbjects having a' perimeter smaller than a specified 
• ' length, where thd-thVeshold length may be predetermined empirically. 
- ■ 50 ■ ■ ' [0040] Once the' objects' h^ve Beeh located, as described with respect to step i 00, the next general step, step 200, 
- is to mod^l the shape of the object. For purposes of siniplicity "the following description wilt treat rectangular-shaped 
i ' bbjects, however, it wilrbe^ppreciated that the description is extensible to other polygons and even to shapes having 
portions thereof represented by curves (e.g., circular or elliptical objects) . The result or out|Dutfr^ 
erably a set of edge traces, in the form of linked lists. that identify bounding pixels abdijt each object within the scanned 
' ' ' 35 ^ image. These traces can be used to extract each ^object, but orientatioh is not yet determined! To improve the quality 

' - ■ of th^ object extractioh, the bbjiect traces are fitted to a nlodel shape. Orientation information, etc., may then be extracted 

■ f rorh the fitted parameters. In the described embodiinent the object traces are fit ta a' rectangular model, however, other 
^ ■ ■ ■ ish^pesarfe' possible: ^^'"'^^ '^^^ ^'^'-'^ ■■■■■ - ■' ' '-y- ' " 

' ■ ' " ' [0041 One methdd'df ^fitting the edge'traces to a rectangular shape is a least-squares approach to fit to a rectangle, 

'v: 40 ■ fo accdriiplish tHe least-sdu'aresfittingj'the edge trace is firsit decbmpdsed into four sets of points; each corresponding 
''-'to bnebf the four isjdes'df the rectahgular object: The decomposition into four sets of points can be accomplished in 
' ' ' ■ ^'^seiveral Ways siis^debdr'^ ■ ^ ^ ^. . ; 

[0042] The first method has two principal parts; (ay catiegorizing the edge points into a se! of bins associated with a 
single line, and (b) performing recognition on the bins for rotated shapes. Referring now to Figure 5, vyhere the first 
45 decomposition method is depicted in detail, step 204 calculates the slope at each point along the edge trace. Step 204 
preferably accomplishes the slope angle calculation by performing a linear regression on a small window of neighboring 
edge points? For example, 2 points lying on either side of the'edge point for which the' slope is being determined. The 
angle of the line passing through the centerbf each ppiht is deiernriined using lindar regression in a small window 
centered on each point. Each regression requlreis 4''addit|dns 'per poirit in the Window, plus 2 subtractions, 2 multipli- 
50 cations, and an arctangent calculation, however, the regression algorithm may be further optimized to remove most of 
the addition operations. In a preferred embodiment, which reduces the computational complexity, a sample of the edge 
pixels are employed for slope angle calculations and sorting, thereby reducing the numberdfdalculations necessary 
to categorize the edge pixels. ^ 
[0043] Next, at step 206, the process constructs a list of slope categories or bins. The slope categories are con- 
55 structed for each edge point by calculating the magnitude of the difference In the slope angle Between the current point 
along the edge (e.g., point B in Figure 8) and the prebfedirig point {e.g., point A in Figure 8]. Jf the difference is less 
than the value TOLERANCE (determined empirically to be ± 5 ddgrees In one embodlmerit), then the point is assigned 
to the same slope category as the preceding point, otherwise a new slope category Is created and the point is assigned 
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to it. Referring to Figure 8. the aboye-described prccess would assign points A, B and C to a first siope category, points 
D, E, F. G and H to a second slope category and points 1, J, ... to yet another slope category. Finally; if the slope category 
for the last edge point has approxtnnatelif, the same, slope angle as- the first slope category, then all points within the 
first and last slope categories are joined together-Jnto a single category. , , - - ... 

5 [0044] Once the slope categories are established at step 206, and^stpred in^a data structure.- they are then sorted 
at step 208 and ordered acc9rding-t9 the number^ of edge points^assigned to each category. For, rectangular objects, 
... the top four slope categories, those containing. the jnostjBdge points, should correspond to points along the four edges 
of the rectangle. The top slope categories are ttiea selected at step 210. It will be appreciated that one would use the 
top six categories .for hexagpnal objects,, and sirnitecly the top. three categories for triangular objects, etc. 

10 [0045] Alternatively, steps 20aand 2l0.may be replaced by a:step that processes the slope angle categories or bins 
by simple, or eyen statist icaLe I imination,..whe re iri4hqse categories .with few.entries are renrroved. For example, an 
empirically determined threshold of 5, pixels rriay be.applied so that only bins.Jiaving.rriore.than 5 pixels with a common 
angle are kept. Subsequently an .average angle for a^category. may be deterrr^ined .using .simple linear regression of 
all the points assigned to a particular category. With the average angle determined^ a further refinem.ent of the categories 

IS would be possible, conibining thq?e .categories haying;Substantial!y comnionj.angles..ln. partis each category is 
checked and if. adjacent categorles.are substantially coljjnear, Jhe.^catego^ joined. Thus each of the remaining 

bins or categories represents a^set of collinear pqirits .ly.ing .alon The edge points assigned to each of the 

remaining slope angle, categories represent.the edgp.trace; decomposed intp^the four sides, of the rectangle. It will be 
.appreciated that this.alterpative is^ broadly^ direpted-to. the process of ".filtering" or refining the categories to identify 
those representing the actual edge of the. objects. Accordingly, eqi|iva lent methods of accomplishing the refinement 
of the categories are contemplated. . . 

, . [0046] This first method of-diaracterizing the.object tDouridaries is conn putationally intensive due to the measurement 
. of the average slope at each edge, point.JnJhe alternative embofJiment mentioned. previously.^.to improve speed, the 
edge trace may .be sampled to reduce the totaj number.of points that must, be processed and pategcrized. 

25 ,[0047] It will be further appreciated that Jt may be possible,- from an .analysis of the ordered categories, to identify 
the shape. For exannple, a statistically significant difference in the.number of poirits between a third and fourth category, 
or the complete lack of a forth category, are .indicative of a triangular-shaped object. . 

[0048] . Referring to Figure 6, depicted therein is the second method. by which the object shapes may be modeled. 
After retrieving the edge trace list data at step 202. step 252 calcul3tes the center of mass, of the object. Although there 

30 are a number of vyell-known methods for calculating the center of rpass of the object, in the case of rectangular objects 
a straightfonward approach would be averaging the .(x.y) coordinates of the edge.points. Next, the edge point closest 
to the center of mass would be located at step 254. The closest point-will be the approximate center of the long side 
of the rectangle. Referring again to Figure 8, the angle G from,the center-of-mass (CofM) to the center point (La/2) is 
the approximate .rotation angle (G) of the rectangle. - . 

35 [0049] Once the rotation angle is determined, it is employed in step 256 to deterrpine the.approximate length of the 
. minor axis of the rectangle at step 258. In particular, the distance from. the center-of-mass to the average position of 
all edge points that lie in the angular range O-AA to G+AA is deter;mined. This distance is ah approximate measure of 
one-half the minor axis length of the rectangle. AA is an empirically determined value on^the order of approximately 
5 degrees. Step 260 approximates the length of the major axis (La) :in^much the sarjie manner.-.The distance from the 

40 center-of-mass to the.average position (G+90)+AA is an approximate rrieasure^qf one-hajithe lepgth of the major axis 
^ of the rectangle. Having approximated the orientation angle, ^nd the lengths of the major and minor axes, step 264 
calculaies an angular range (as measured with respect to the cent!er.<>f-n>ass)4qr each side of the rectangle that en- 

,. . compasses only those edge points associated .with that side; , . ■ . p . y . mv^;^ 

,.^b.=;^^^"2(La,LJ half, angle. width of rriajor axis;- ^ ■ - 

. \ r, : ^ *^'a ?P'®b . . half angle.width of minor axis; . . . ^. , • ^ 

' r 9(5= GVT9'- .where 'TOL^.95 to avoid. corners; 
/ ' '9a=^'a*TOL , where.TOL^:0.95^p avoid corners;. 



and , .. . , ^ . 

b)Rangei: (0+8^) to'(G-9i,) 

, ,7f?snge2 . : ({G+9q)+ea).tq (jfi^Qoye^ , 

^'^Rangej: ((e-h18Ci)+Gb)'to ((G4^J8p)-9b) * ' 
Range4: ((9+270')+ea) to ((e+270)r93j . . , 
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Once the angular range is determined, step 266 finds all the edge points that lie within each of the four angular ranges 
(relative to the center-of-mass) deternnined above, thereby identifying the edge points corresponding to each side of 
the rectangle. It will be-app^reciated that this ■technique is less sensitive to edge-noise than the first method described 
above. 

5 [0050] Once the edge trace has been decomposed into four sets of points, each set corresponding to one of the four 
sides of the rectangle, a least squares calculation for fitting the points to rectangle is evaluated at step 280. A rectangle 
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where py=-1 . A least squares fit yields the fitted parameters: 
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where (Xf^f, Yk,] is the edge point of the Jc^ side, and n/^ is the number of edge points associated with the k'^ side. 
Subsequently, once the least squares fit yields the fitted paranneters (p. a^, a^, 03), they are converted at step 282 
the into four coordinate pairs marking the corners of the rectangle. Moreover, the rotation angle of the rectangular 
object is accurately represented by the slope parameter p. 

[CM)51] Yet another alternative method for fitting the edge traces to a shape is a method employing binary moments 
for fast Image bounding. Using the binary mask generated as described with respect to step 106 (e.g., the adaptive 
seed algorithm), or alternatively with a simple thresholding operation, the image is rendered in a binary bitmap form 
where each pixel value is a 0 or 1 indicating background or non-background regions. Once the borders are detected 
for an object using the binary mask, the alternative embodiment depicted in Figure 7 employs second-order binary 
moments to tit a shape (e.g., rectangle) to the object. 

[0052] Referring to Figure 7, depicted therein Is a generalized flowchart representing the steps of the binary moment 
boundary finding technique. At step 100, the object edges are located and recorded as previously described, thereby 
providing as an input a linked list of boundary or edge pixels referred to as an edge trace, step 290. Using the boundary 
list, the second order moments are calculated (slep 292) in an efficient manner using the equation: 



where piij) is the image pixel value at image coordinates (i.j) and p^(i) Is the order moment of the scan line. 
Because the object boundary pixels are previously determined, the process can be simplified and the rightmost and 
left-most boundary pixels for a particular scanline are used for the 1 st order (absolute) moment calculations. 
[0053] Subsequently, the 2nd order (central) moments (mpQ, rriQ^, m^^, m^p. and m^) are calculated using the 
1st order moments and the following equations: 



[0054] Having determined the 2nd order moments, they are employed to characterize an ellipse and from the ellipse 
the bounding box about the object, step 294. In particular, the center of the ellipse (x. y), the lengths of each axis (a 
and b) and the rotation angle (G) are determined. The bounding box for the rectangular object is determined as a 
rectangle centered at (x.y) with sides of length 2a and 2b, rotated by an angle 0. While this renders a bounding box 
slightly larger than the object, this is done so as to provide a safety margin for the calculation, and to avoid cropping 
a portion of the object. If a tighter bounding box is desired, the rectangle would be characterized with sides of length 
2aa and 2ab, where a Is set equ^ to V3/2 or a slightly smaller value to accomplish edge trimming or cropping (e.g., 
on the order of one or more pixels). 

[0055] After each object has been modeled as a shape (e.g., rectangle), a structured image is created as described, 
for example, in US-A-5.485,568 to Venable et a!. The structured image consists of one "child" structured image for 
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each object detected using one of the nnethods described above. The 'structured image definition contains attributes 
'that specify whicK'fectahgle of the scanned image cohtaihs'the'bbject data; and also the rotation angle required to 
' correct forany orierltatioh skewyhgure 9'is'an example'df a structured'imagecreited ih accordance with the previously 
described processes, the structured image containing a pair of rectangular-shaped image objects. 
'[0056] " In one'embb'dimerit (^ ihe present iniveht ion;- ddpicted in Figure'10, the^structured-image is designed such 
thaVWH eh Teti delved; all ofijects -arede-rbtatedancl Idid otit ih a grid fashion; In particular, Figure 10 illustrates a user 
interface 400 thaf-rriaV be enipioyed'with varioOs aspects df fhe 'previously''describecj object shape recognition method 
to-'provide -an'-ihteHigeht or -'snnaft'' j3laten' scJ^rihirig system^^^ -smart scafTning system' represented by Figure 10 
preferably provides; § means by whicH a us (§rxar> interface W k digitizing scanner to efficiently obtain digitized rep- 
resentations of objects placed on platen 24- of a' s^canher;^^^^^ ■ ■ ' ' ' ■ ■ ■ . - ■ , - ■ 
[0057] - Fdr exairiplevfeferr'ihg to Figure' 'iWh^ place a nOhnber of photographs 
oh' "the *scanh Si' platen: Once 'placed thereoh, the -user iriay theh an 'ope ration from vegion 410 of Figure 10 to 
^ cause the completer systerh 22 to initiate scanhirig by"scanher 26. Although not specifically shown,* It will be appreciated 
that-Various methbds for- initiating the cligitizMtion of the objectsmay be employed, including; but not limited to, keyboard 
' ehtry, tbiJc'h'-sensitive'sbreeh'selectio depre^sibh'bf'a remote switch or even triggering of a platen sensor by a user 
closir1g%ie |:iIateh-6cJ(/^^As^^^^ (412)br equivalent scanning selection Is 
made, system 20 scans the objects placed on platen 24 and temporarily stores the data in the file using the information 
reflected in region 420 of the user interface screen. For example, the various image objects (A, B, C and D) may be 
found within an image as illustrated in Figure 11. Once the image is scanned, it is analyzed"as'described above to 
identify the image objects. The image objects may then be manipulated by the smart scanning system to automatically 
orient and position the images, for example they may be autoriiaticaliy placed in a predefined template and rendered, 
such as the representation depicted in region 430 of the user interface. It will be appreciated that a user may also be 
-givesn addftiohal edit 'capabiliiy' with respedt Id t^ terhplat^l'fbr*ex^rriple/td add captions to the objects or to include 
titles 432 and subtitles 434 as illustrated: Ihputfor'such t^xt-based'e^ be accomplished via the user interface 
■ options- dejDictGd in region -440. . ' :: , ;c . ^.r- - ; ' .^ , 

[0058]' 'It will' be appreciated by those skilled in the art that by defining the boundaries of the objects in an image 
accurately'^ derot^tidn operation may be employed to more accurately deskew the object. For further illustration of 
this advantage reference may be had to Figure 12. Depicted therein is an object O within an image segment I. In a 
typical deskewing operation, 'the bilhogonal boundaries of the image are determined ^asi illustrated by bounding box 
- 50Q-and1he image' is deskev^ed fb' prcducei im^ge indicated as '504 including the object O'. It will be appreciated 
that in order to 'deskew the' liriage that 'is not clearly defined by boundaries, the entire irtiage area must be rotated, 
leadingto both a larger dfeskewed iiria^^ area and fheheedf o create pixels to fill in regions 508. The present invention, 
by accurately defining the boundaries and orientation of the object^ avoids the need to incorporate regions larger than 
the object in the derotation operation. Thus, the pixels representing the object itself are the only, pixels that are derotated 
tO'produce the G' output image. . . 

[0059] ' Also enabled by the smart scanning system would be image editing bapabilities' as illustrated in region 450 
of the user interface. Having identified each of the objects within the'image, it is possible to isolate the objects, create 
separate images therefrom, and to then individually process the images. Thus the individual image objects automati- 
cally jDtaced with iri the terhplate of region 4'3b may be individually selected' nrianipula'ted, scaled (button 452), rotated 
(button 454) or cropped (button 456). It will be appreciated that the scaling, rotation' and cropping operations are in 
addition to those which are preferably automatically applied by the system as the result of the previously described 
"object Teragmti6n7t^6t^^ '* ^' — ■ : r. ■ r'.: 

[0060] For example, the image scaling button, illustrated with cross-hatching to depict selection, will allow the user 
to move a cursor (not shown) to select an object (e.g., image object D) and then to drag a side or corner of the object 
"sb'as ttf scaie the lnna'ge o^^ facilitate the editing of the object^, "iiontror pbi those illustrated about 

the boundary of image object D (436) may be ehripioydd in a manner weir-l<nown to those who 'design user interlaces. 
[0061] As noted, a predefined template may be used to automatically "place" image objects. in relative positions on 
a docunriehtorpagelher^df. It will be appreciated that such terhplates may be in the form of a structured image definition, 
so that the template can be used to specify a different layout for the structured image to be generated. Thus, a family 
seeking to'fjti^it^'photograpihs in a "iigitarphdtb 'ablirh"" 'may be able to create a tenii plate describing a page similar to 
that shown in region 430 of the user interface. The template would theh be used tb'autdrhatically organize individual 
images or plural objects within a larger document image. 

-'[0062] '-^'fh'a'preferreid ennbodihnent, the output of the sniahsbahhihg system would be 'a structured image document 
forrhat ias"d6scrib^^^^ et aL ih US-^A-5;i85,566: Ah impo^^^ of strLictufed images is the ability 

to store image processing operations in their description. This means that the structured image can contain image 
processing operations other than simple object deskewing attributes. For exampje, autonnatic image enhancement 
operations nnay be included within the structured irnWge^ s'uch thm'tHte objects identified can'be individually enhanced. 
Alternatively, the smart scanning system output may take the form of a digital document in one of many well-known 
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page description languages. , . . \ 

[0063] Once the "page' composed in window 430 in the condition desired by the user, the user may save the image 
by selecting the "Save Edited Image" buttpn 460.; More importantly, a user may then print or otherwise distribute the 
composed page (s).^ • . ^ „ . , „ . 

5 [0064] Although the various embodiments of the^ present invention have. been described with.respect to the smart 
scanning system, it will be appreciated that the acquisition of images, and the.printing and distribution of the composed 
pages can be accomplished via^netwprks or on a walk-up digital.copier For example, a user. rpay have photographs 
. automatically. scanned by a film prccessor, and a. digitized starnp sheet sent to the.user.yia.a network. The stampsheet, 
beirig in a structured image. format qould then be prccessed using the. smart scarining systeni to produce pages of a 

10 digital photo album with one or more objects.pn each page.. . . - , , . - r 

[0065] In recapitulation, the present inyentioais an intelligent scanning apparatus for processing a digital input image 
to automatically characterize a plurality of qbjects therein, and to employaiie characterization as the basis for rudimen- 
tary image editingpperations ?o as to produce a digital dpcurpent. In thejdigital document, the objects may be derotated, 
shifted, cropped or otherwise aligned in. a. predetermined fashipn.in.accordance-^with a template. The scanning appa- 

75 ratus of the present invention not onfy enables the. scanning of a plurality.ot obje.c^^ in.an intelligent manner 

so as.to enalDle turther processing and manip.ulation of-^the images .asscxiated.yyith l\)e objects. 
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Claims 

1. ^An imaging apparatus, including: 



an image input device (2^), said irpage input device producing -a digitized, image including representations of 
each of a plurality of objects inriaged by said device: : . p. ^ x - ^ ^xm 

a programmable computer (20) capable of processing the.digitized Jrnagp, said computer including a first 
nriemory (52) for storing at least a portion of the digitized image and prograrp memory. (52) for the storage of 
executable code suitable for causing said computer to execute irnage prpcessing operations on the digitized 
image, „ . , . . 

said computer, in accordance with, preprogrammed instructions, .identifying. the plurality of objects within the 
digitized input image, modeling shapes representing boundaries of, each of the plurality of objects, and char- 
acterizing each of the plurality of objects by parameters. including shape, positiori and orientation; and 
said computer. automatically composing an output document including a, representation of at least one of the 
, plurality of objects. . . . . 

35 2. The apparatus of claim 1 further including a user interface (34) Jor displaying a representation of the output doc- 
ument incorporating the representation of at least one of the plurality of objects wherein one of- the parameters of 
. the at least one object has been altered. , . .. , ^ . , ' . 

3. ' The apparatus of claim 1prclaim2,.whereinthe posit ion of the representation of ^he object ^ 
40 positipn in the Qutput document. . . . ' , ■ ; . /..'"J . 

4. The apparatus of claim 2. wherein only the representation of the object .is,?derotated so as pjace a derotated rep- 
^ resentatipn of the pbject in the output document. ^ - ;\ n -- 

. . .5. ,.The,apf3aratus of any of the preceding clairris, wherein said image input device, Jncj.gdes a surface (24) upon which 
y , the plurality ,Qf objects a re placed for digltizati.^^ . , . . 

, . .6. The apparatus of any of the preceding claims, wherein the first rriemory, is a frame buffer. . - 

^? f. : jT.vr? pPP^r^^us of any of the preceding claims, wherein the first memory, is a datH Sjtorage media-accessible by said 
. ^ . u. 'programmable computer. . . . , . - _ ' . ' . .. . . ^ . - // 

. . 8. . ^ The apparatus, of any of.the pfeceding claims..furth template me n^ory. for, storing, at least one template 

. for .control ling the position and, prientatiop of the representation of the at least .one object, during composition of 
55 ^ the output document. . y * , , . ... ...i, 

9. The apparatus of claim 8, wherein.the at least orie template is stored' in the form of a structured innage. 
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10. The apparatus of any of the preceding claims, further including: 

' !5 -y^^; ' ^ "user interface, said user iriteilace cbrtipfisliiQB user input device (30) responsive to a user selection and a 
display (34) to depict the output document in conjunction with user selectable options; 

5 wherein said computer further include's^preprbgrammki'instructibhs to update the display in accordance with 

a selection of the user selectable options'. ' ' • ' ■ "' ^ ' 

11. The apparatus of claim 10, wherein^the prbgrarnmable computer,' ih' conjunction with the display, provide user 
selectable options to: . ' ' j i .i- s ; ' '. ■ ■■ .■■ ^ " 

10 ■ '^^^--^ ^'^^.'^ ' v;--'^--..: , . : 

select at least 'dne- object in the output- dbcum^ ^ 
alter at least one parameter-of the selected^ bbject;" and - ' ' 

siore, in memory, a version of the output document after a user has altered a parameter of at least one object 
therein. 

75 

1 2. The apparatus of claim 1 0 or claim 1 1 , wh erein the user selectable options are selected from the g roup consisting of: 

inserting textual content into the output document; defining a storage location for the output document; storing 
the output document; 
20 scaling the representation of the at least one object; 

rotating the representation of the at least one object; 

cropping the representation of the at least one object; 

altering the colorization of the representation of the at least one object; and 

enhancing the representation of the at least one object. 

25 

13. A digital copying apparatus, including: 

an image input device (26) having a transparent platen (24), said image input device producing a digitized 
image including representations of each of a plurality of objects placed on the platen; 
a programmable computer (20) capable of controlling the operation of the digital copying apparatus and of 
processing the digitized image, said computer including a first memory (52) for storing at feast a portion of the 
digitized image and program memory (52) for the storage of executable code suitable for causing said computer 
to execute image processing operations on the digitized image, said computer, in accordance with prepro- 
grammed instructions, identifying the plurality of objects within the digitized input image, modeling shapes 
representing boundaries of each of the plurality of objects, and characterizing each of the plurality of objects 
by parameters including shape, position and orientation, and said computer automatically composing an output 
document including a representation of at least one of the plurality of objects; 

a user interface, said user interface comprising a display (34) to depict the output document and a plurality of 
user selectable option, wherein said computer further includes preprogrammed instructions to update the dis- 
play in accordance with a selection of the user selectable options; and 

a printing engine to produce, in accordance with the output document, a substrate bearing marks in accordance 
with representations contained in the output document. 

14. The digital copying apparatus of claim 13, wherein the digitized image is a color image and wherein the marks 
45 placed on the substrate are marks of at least two distinct colors. 

15. The apparatus of claim 13 or claim 14, wherein said user interface comprises a user input device (30) responsive 
to a user selection and a display to depict the output document in conjunction with user selectable options; and 

wherein said programmable computer further includes preprogrammed instructions to update the display in 
accordance with a selection of the user selectable options. 

16. The apparatus of claim 15, wherein the programmable computer, in conjunction with the display provide user 
selectable options to: 

55 * 

select at least one object in the output document; 
alter at least one parameter of the selected object; and 

store, in memory, a version of the output document after a user has altered a parameter of at least one object 
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therein. ^ . . . 

1 7. The apparatus of claim 1 5 or clatrn J 6. wherein the user selectable options are selected from the group consisting of: 

5 inserting textual content into the output document: ^ , ; - - 

defining a storage location for the output ciocument:- , . - 

storing the output document; 
scaling the representation o| the at Jeast-.one object; 

rotating the representation of the at least one object; . ^ . - . . 

^0 cropping the representation of the at least one object; 

altering the colorizatton of the representation of the at least one object; and . . 
enhancing the representation of the atJeast, one object. , >. , ^ 
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